Neurally-Guided Procedural Models: Learning to Guide Procedural Models with Deep Neural Networks

We present a deep learning approach for speeding up constrained procedural modeling. Probabilistic inference algorithms such as Sequential Monte Carlo (SMC) provide powerful tools for constraining procedural models, but they require many samples to produce desirable results. In this paper, we show how to create procedural models which learn how to satisfy constraints. We augment procedural models with neural networks: these networks control how the model makes random choices based on what output it has generated thus far. We call such a model a neurally-guided procedural model. As a pre-computation, we train these models on constraint-satisfying example outputs generated via SMC. They are then used as efficient importance samplers for SMC, generating high-quality results with very few samples. We evaluate our method on L-system-like models with image-based constraints. Given a desired quality threshold, neurally-guided models can generate satisfactory results up to 10x faster than unguided models. CR Categories: I.3.5 [Computer Graphics]: Computational Geometry and Object Modeling—Geometric algorithms, languages, and systems G.3 [Probability And Statistics]: Probabilistic algorithms (including Monte Carlo) I.2.6 [Artificial Intelligence]: Learning—Connectionism and neural nets